# Web crawler

PodCrawlerMCP is an MCP service that discovers podcast content through web crawlers, helping AI assistants find podcast shows and episodes based on topics.

DuckDuckGo search API service, providing multiple search functions through the MCP protocol

The Crawl4AI MCP Server is a Python-based web crawler project that integrates Playwright browser automation, the DuckDuckGo search engine, and the MCP protocol SDK for intelligent web data collection.

The Firecrawl MCP Server is a web crawler and data extraction service based on the Firecrawl API, providing functions such as web page scraping, content search, site crawling, and structured data extraction.

A client project based on Python 3.13 that integrates MCP services and the GPT-4 model, providing interactive tool invocation and network search functionality.

mcp - server - webcrawl is an advanced web crawler data search and retrieval tool designed specifically for AI clients. It supports multiple crawler formats (such as WARC, wget, etc.), provides full - text search, Boolean logic queries, and resource type/status filtering functions. It can be seamlessly integrated with Claude Desktop, is installed via Python, and is suitable for tasks such as building website knowledge bases or conducting SEO/performance audits.

The Bocha AI Web Search MCP Server provides a search service for Chinese Internet content that complies with Chinese regulations, supporting output in Markdown and JSON formats.

An example of MCP service for price research on the Cimri website using Puppeteer